Seeing with your hand

ABSTRACT

The disclosure describes methods and systems for gathering and conveying information, for example, such as with a hand of a user. In one embodiment, the method may include using a detector to record a series of images of an environment and detecting a predetermined motion by comparing two or more images in the series. The method may include selecting a function based on the predetermined motion and triggering the function. In another embodiment, the method may include using a first detector to record a first series of images of an environment and using a second detector to record a second series of images of an environment. The method may include detecting a predetermined relative motion by comparing one or more images from the first series with one or more images from the second series, and selecting and/or triggering a function based on the predetermined relative motion.

BACKGROUND

Humans naturally use their hands and fingers as a means of gatheringinformation. For example, when moving in a dark environment, humansnaturally put their hands out to gather information about theirsurroundings by touch. Similarly, when a small object is lost, forexample, underneath a couch, humans naturally put their hands under thecouch to locate the lost object by touch. While gathering information bytouch is in some cases an acceptable substitute for seeing, in manysituations it may be desirable to “see” the inaccessible environment tobetter gather information.

In addition to gathering information, humans also naturally use theirhands and fingers to convey information. For example, when givingsomeone directions to a location, many humans naturally use one or morefingers to point towards the location. Similarly, when discussing aparticular object, many humans naturally use one or more fingers topoint to the object. However, typically the amount and types ofinformation that can be conveyed by a human's hands is limited tospecific contexts.

SUMMARY

Devices and methods for gathering and conveying information with a handare described. In one example, to allow a user to “see” an inaccessibleenvironment, a user may wear a device equipped with one or moredetectors on one of his or her hands, or on other areas of the user'sbody. The detector(s) may record a series of images of the environment,and a display may display the series of images to the user. In thismanner, the user may “see” the inaccessible environment as if the user'seyes were located on the user's hand or finger, or on the other area ofthe user's body. Alternately or additionally, motion of the detector(s)may be determined by comparing images in the recorded series of images,and one or more predetermined motions may detected among thepredetermined motion. One of more functions may be triggered based ondetection of the one or more predetermined motions.

In an embodiment, a glove is provided that may include a detectorpositioned on a fingertip of the glove. The detector may be configuredto record a series of images of an environment. The glove mayadditionally include a processor configured to determine at least onepredetermined motion of the detector by comparing at least two images inthe series of images and, based on the at least one predeterminedmotion, select at least one function from a plurality of functions.Additionally, the glove may include an output interface configured totrigger the at least one function.

In another embodiment, a wearable device is provided that may include adetector configured to record a first series of images of anenvironment. The wearable device may also include and a processorconfigured to stabilize the first series of images to produce a firststabilized series of images, determine at least one predetermined motionof the detector by comparing at least two images in the first stabilizedseries of images, and, based on the at least one predetermined motion,select a first at least one function from a plurality of functions. Thewearable device may additionally include an output interface configuredto transmit an output based at least in part on the first at least onefunction.

In another embodiment, a method may include recording a first series ofimages of an environment with a first wearable detector, stabilizing thefirst series of images to produce a stabilized first series of images,and determining at least one predetermined motion by comparing at leasttwo images in the first stabilized series of images. The method mayfurther include maintaining a first set of correlations between aplurality of predetermined motions and a first plurality of functionsand identifying the at least one predetermined motion in the first setof correlations so as to determine a first at least one functionassociated with the at least one predetermined motion.

Other embodiments are described below. The foregoing summary isillustrative only and is not intended to be in any way limiting. Inaddition to the illustrative aspects, embodiments, and featuresdescribed above, further aspects, embodiments, and features will becomeapparent by reference to the figures and the following detaileddescription.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 shows an overview of an embodiment of an example system.

FIG. 2 shows a block diagram of an embodiment of an example device.

FIGS. 3 a-3 b show an example application of an example device.

FIG. 4 shows a block diagram of an embodiment of an example device withadditional logic processing functions.

FIGS. 5 a-5 e show an example application of an example systemimplementing motion-detection logic.

FIG. 6 shows a block diagram of an embodiment of an example device withmore than one detector.

FIGS. 7 a-7 d show an example application of an example system detectingrelative motion with two detectors.

FIG. 8 is an example block diagram of a method of operating a device, inaccordance with at least some embodiments described herein

FIG. 9 is an example block diagram of another method of operating adevice, in accordance with at least some embodiments described herein

DETAILED DESCRIPTION

The following detailed description describes various features andfunctions of the disclosed systems and methods with reference to theaccompanying figures. In the figures, similar symbols typically identifysimilar components, unless context dictates otherwise. The illustrativesystem and method embodiments described herein are not meant to belimiting. It will be readily understood that certain aspects of thedisclosed systems and methods can be arranged and combined in a widevariety of different configurations, all of which are contemplatedherein.

1. Overview of a Device and System

FIG. 1 shows an embodiment of an example system. As shown, the system100 includes a wearable apparatus 102 including a detector 104 and aprocessor 106. The system 100 also includes a display 108.

The size and shape of the wearable apparatus 102 shown in FIG. 1 aremerely illustrative and are not meant to be limiting. Other sizes andshapes of the apparatus 102 are possible as well. As an example, whilethe wearable apparatus 102 is shown as a glove, it is to be understoodthat the wearable apparatus 102 is representative of any number ofwearable apparatuses. In some embodiments, the wearable apparatus 102may be wearable on a hand, wrist, or finger of a user. The wearableapparatus 102 may be worn on other areas of a user's body as well, suchas a foot, hip, or back of a user. Other areas of a user are possible aswell. Examples of wearable apparatuses include wristbands, watches,rings, and mittens. Other wearable apparatuses are possible as well.

Similarly, the size, shape, and location of the detector 104 shown inFIG. 1 are merely illustrative and are not intended to be limiting.Other sizes, shapes, and locations of the detector 104 are possible aswell. As an example, while the detector 104 is shown to be located on afingertip of an index finger of the wearable apparatus 102, otherlocations of the detector 104 are possible as well. For instance, thedetector 104 could be located on a different fingertip or elsewhere onthe wearable apparatus 102. The wearable apparatus 102 may include oneor more additional detectors as well.

Similarly, the size, shape, and location of the processor 106 shown inFIG. 1 is merely illustrative and is not intended to be limiting. Othersizes, shapes, and locations of the processor 106 are possible as well.As an example, while the processor 106 is shown to be located on thepalm area or the back of the glove 102, other locations of the processor106 are possible as well. For instance, the processor 106 could beintegrated into the detector 104 on the wearable apparatus 102, or couldbe located elsewhere on the wearable apparatus 102. Alternately, theprocessor 106 could be located remote from the wearable apparatus 102,such as on another wearable apparatus worn elsewhere on a user's body,on an apparatus not worn by the user, or as stand-alone device.

Similarly, the size, shape, and location of the display 108 shown inFIG. 1 are merely illustrative and are not intended to be limiting.Other sizes, shapes, and locations of the display 108 are possible aswell. As an example, while the display 108 is shown to be located inproximity to the wearable apparatus 102, other locations of the display108 are possible as well. For instance, the display 108 could beintegrated into the wearable apparatus 102. Alternately, the display 108could be located remote from the wearable apparatus 102, such as onanother wearable apparatus worn elsewhere on a user's body, on anapparatus not worn by the user, or as stand-alone device.

In general, the detector 104 may be positioned on the wearable apparatus102 to have a line of sight to an environment surrounding the wearableapparatus 102. The detector 104 may be configured to record one or moreseries of images of the environment surrounding the wearable apparatus102, and thus may include a camera. The detector 104 may also beconfigured to transmit the recorded series of images to the processor106. To this end, the detector 104 may be communicatively coupled to theprocessor 106 by one or both of a wired link and a wireless link.

The processor 106 may include one or more processing elements such asone or more processors, data storage, and an output interface. Theprocessor 106 may be configured to perform various types of imageprocessing and analysis on the series of images received from thedetector 102, as described below in connection with FIGS. 2, 4, and 6,for example. The processor 106 may additionally be configured to receivethe series of images from the detector 104 and to transmit to thedisplay 108 an output based at least in part on the series of images. Tothis end, the processor 106 may be communicatively coupled to thedisplay 108 by one or both of a wired link and a wireless link.

The display 108 may be configured to display the output received fromthe processor 106. The display 108 may additionally be configured todisplay information received from one or more additional sources. Thedisplay 108 may be, for example, a heads-up display, a head-mounteddisplay, a flat-panel display, a light-emitting diode (LED) display, anelectroluminescent display (ELD), a liquid crystal display (LCD), anorganic LED (OLED) display, or any other type of display now known orlater developed.

2. First Example Device

FIG. 2 shows a block diagram showing an embodiment of an example device.As shown, the device 200 includes a detector 202, a processor 204, datastorage 206 in which is stored stabilizing logic 208, and an outputinterface 210. The elements of the device 200 are shown coupled by asystem bus or other mechanism 212.

It is to be understood that, while each of the detector 202, theprocessor 204, the data storage 206, the stabilizing logic 208, and theoutput interface 210 are shown to be integrated into the device 200, thedevice 200 may, in some embodiments, comprise multiple devices amongwhich the elements of device 200 are distributed. As an example,detector 202 may be separate from (but communicatively coupled to) theremaining elements of device 200. As another example, detector 202,processor 204, and output 210 may be integrated into a first device,while data storage 206 and the stabilizing logic 208 may be integratedinto a second device that is communicatively coupled to the firstdevice. In this example, the device 200 may comprise the first deviceand the second device. Other examples are possible as well.

The detector 202 may be, for example, a camera or other imaging device.In embodiments where the detector 202 is a camera, the camera may beconfigured to detect visible light, or may be configured to detect lightfrom other portions of the spectrum, such as infrared or ultravioletlight, or x-rays. Other types of cameras are possible as well. Thedetector 202 may be a two-dimensional detector, or may have athree-dimensional spatial range. In some embodiments, the detector 202may be enhanced through sensor fusion technology. In any case, thedetector 202 is configured to record a series of images of anenvironment.

The processor 204 may be or may include one or more general-purposeprocessors and/or dedicated processors. The processor 204 may beconfigured to perform one or more types of image processing and analysison the series of images recorded by the detector 202 so as to produce anoutput. In particular, the processor 204 may be configured to executestabilizing logic 208 to stabilize the series of images.

While images gathered by a human's eyes are naturally stabilized byseveral reflexes of the human's eyes and brain (e.g., thevestibule-ocular reflex), the series of images recorded by the detector202 may not be stable as a result of movement of the user and detector202 during recording. Thus, it may be desirable in some embodiments forthe processor 204 to stabilize the series of images.

To this end, the processor 204 may be configured to execute stabilizinglogic 208 stored in the data storage to stabilize the series of imagesto produce a stabilized series of images. In general, the stabilizinglogic may produce the stabilized series of images by modifying orremoving one or more images in the series of images based on a detectedorientation and/or motion of the detector 202. To this end, the device200 may, in some embodiments, additionally include an orientation sensorsuch as a gyroscope or accelerometer to detect the orientation and/ormotion of the detector 202. Other types of stabilization are possible aswell, either instead of or in addition to the stabilizing logic,including lens-based stabilization (in which a floating lens element iscontrollable by electromagnets to move orthogonally to the optical axisof the lens to account for detected movement of the camera) andsensor-shift stabilization (in which a sensor capturing the image ismoved by an actuator to account for detected movement of the camera). Inthis example, the output may be the stabilized series of images.

In some embodiments, the detector 202 may be configured to performoptical and/or lens-based stabilization on the recorded series ofimages. This logic may be partially or wholly integrated into thedetector 202, and may be used instead of or in addition to thestabilizing logic 208. In this example, the output may be the stabilizedseries of images.

The processor 204 may be configured to perform other types of imageprocessing, correction and analysis as well, and the output may befurther based on one or more of the other types of image processing,correction and analysis.

The output interface 210 may be configured to transmit the output to adisplay, such as the display shown in FIG. 1. To this end, the outputinterface 210 may be communicatively coupled to the display through awired or wireless link. Upon receiving the output from the outputinterface 210, the display may display the output to a user. Through thedisplay, the images seen by the detector 202 can be viewed by the user.

In some embodiments, the device 200 may also include a power supply,such as a battery pack or power adapter. In one embodiment, the device200 may be tethered to a power supply through a wired or wireless link.Other examples are possible as well.

The device 200 may include elements instead of and/or in addition tothose shown. For example, the device 200 may additionally include aflash or other illumination mechanism that is integrated into orseparate from the detector 202. As another example, the device 200 mayinclude one or more interfaces for communicating with one or moreservers, hosts, or remote devices. Other examples are possible as well.

FIGS. 3 a-3 b show an example application of an example device. Inparticular, FIGS. 3 a and 3 b show a user using the example device to“see” a microchip having a number of small, fine detailed features. Theelements of the device shown in FIGS. 3 a and 3 b may be similar to theelements described above in connection with FIG. 2.

In FIG. 3 a, a user's hand is shown wearing a device 300. The device 300may use a detector 302 located on the device 300 to record a series ofimages of an environment that includes the microchip. The detector 302may transmit the series of images to a processer (not shown), such asthe processors shown in FIG. 1 or 2, for example. In some embodiments,the processor may be integrated into the device 300. In otherembodiments, the processor may be located remotely from the device 300.

Upon receiving the series of images, the processor may perform one ormore types of image processing and analysis on the series of images. Asan example, the processor may execute stabilization logic to stabilizethe series of images to produce a stabilized series of images, asdescribed above.

The processor may then transmit an output based on the stabilized seriesof images to a display. In this example, the output 308 may comprise thestabilized series of images.

In FIG. 3 b, a display 306 is shown displaying the output 308 receivedfrom the processor. By viewing the output 308 on the display 306, theuser may “see” a representation of the environment that is “seen” by thedetector 302.

In embodiments where the processor performs additional types of imageprocessing and analysis, the output 308 may comprise additionalinformation, such as measurements of the microchip or a circuit diagramof one or more components of the microchip. Other examples are possibleas well. The additional information may be provided to the display bythe processor or by another device.

In this manner, a user may “see” an environment that includes smalldetails that may not be visible to a user. In other applications, a usermay use the device to “see” environments that are similarly or otherwiseinaccessible. Other applications are possible as well.

3. Second Example Device

While the foregoing discussion has focused on a user using a detector togather information, in other examples, a user may use one or moredetectors to convey information. This may be further explained inconnection with FIG. 4.

FIG. 4 shows an embodiment of an example device with additional logicprocessing functions. As shown, the device 400 includes a detector 402,a processor 404, and data storage 406 in which is stored stabilizinglogic 408, motion-detection logic 410, function logic 412, andcorrelations 414. The device 400 additionally includes an outputinterface 416 and a motion sensor 418. The elements of the device 400are shown connected together by a system bus or other mechanism 420.

It is to be understood that, while each of the detector 402, theprocessor 404, the data storage 406, the stabilizing logic 408, themotion-detection logic 410, the function logic 412, the correlations414, and the output interface 416, are shown to be integrated into thedevice 400, the device 400 may be, in some embodiments, made up ofmultiple devices among which the elements of device 400 are distributed,as described above.

The detector 402 may be any detector configured to record a series ofimages of an environment. The detector 402 may be, for example, any ofthe detectors described above.

The processor 404 may be or may include one or more general-purposeprocessors and/or dedicated processors. The processor 404 may beconfigured to execute one or more sets of logic stored in the datastorage 406.

In particular, the processor 404 may be configured to executestabilizing logic 408 to stabilize the series of images, as describedabove. To this end, the device 400 may additionally include one or moremotion and/or orientation sensors (not shown), such as an accelerometerand/or a gyroscope.

Additionally, the processor 404 may be configured to executemotion-detection logic 410 to detect one or more motions of the detector402. To this end, the device 400 may additionally include one or moremotion and/or orientation sensors, which may be the same as, in additionto, or in place of the motion and/or orientation sensors described abovein connection with the stabilizing logic 408. The processor 404 may befurther configured to execute the motion-detection logic 410 to detect,among the detected motions, one or more predetermined motions or definedmovements or gestures of the detector 402 by comparing one or moreimages recorded by the detector 402. The processor 404 may be furtherconfigured to execute the motion-detection logic 410 to detect a speedof the detected motions. Details of an example operation of themotion-detection logic are described in connection with FIGS. 5 a-5 d.

In some embodiments, the predetermined motions detected using themotion-detection logic may include linear motions, such as upwardmotion, downward motion, leftward motion, rightward motion, forwardmotion, backward motion, and no motion. In some embodiments, thepredetermined motions may include circular motions, such as arcs andcurves. In some embodiments, the predetermined motions may include acombination of motions, such as back-and-forth and start-stop-startmotions. In some embodiments, the predetermined motions may include oneor more swipes, shapes and/or characters, such as symbols or letters ofthe alphabet. Other predetermined motions are possible as well. Anexample of detecting a predetermined motion is described below inconnection with FIGS. 5 a-5 d.

The processor 404 may be further configured to execute function logic412 to trigger one or more functions upon detection of particularpredetermined motions. To this end, the function logic 412 may beexecutable by the processor to look up one or more detectedpredetermined motions in a set of correlations, such as correlations414. Correlations 414 may be correlations between a plurality ofpredetermined relative motions and a plurality of functions. By lookingup the one or more detected predetermined motions in the correlations414, the processor 404 may determine one or more functions that arecorrelated with the predetermined motion(s).

In some embodiments, the functions may be tasks or subroutines that maybe carried out by the processor 404 and displayed on a display or in oneor more applications running on a display. One example of a function maybe entering one or more characters in an application displayed by thedisplay, such as a word processing, spreadsheet, or email application.Another example may be page zooming, such as zooming in or out on thedisplay or on an application running on the display. Yet another examplemay be page scrolling and panning, such as scrolling up or down orpanning left or right on the display or on an application running on thedisplay. Still other examples may be moving an indicator on the displayand selecting an object displayed by the display. Yet another examplemay be using image recognition to identify at least one object in theenvironment and displaying information related to the at least oneobject on the display. In other embodiments, the functions may be tasksor subroutines that may be carried out by the processor 404 but notdisplayed on the display. One example may be storing the recorded seriesof images in data storage. Other functions are possible as well.

In some embodiments, user-friendly correlations may be made betweenpredetermined motions and functions. As an example, motion of thedetector to the left may be correlated with a function of panning to theright on a display. As another example, no motion of the detector may becorrelated with a function of using image recognition to identify anobject located substantially in the center of one or more images in aseries of images and displaying information related to the object. Otherexamples are possible as well.

Upon determining one or more functions that are correlated with thepredetermined motion(s), the processor 404 may trigger the one or morefunctions by sending instructions to the display via the outputinterface 416. To this end, the output interface 416 may becommunicatively coupled to the display via a wired or wireless link. Anexample of triggering a function based on detection of a predeterminedmotion is described below in connection with FIG. 5 e.

While the correlations 414 are shown to be stored in the data storage406 of the device 400, in some embodiments the correlations 414 may bestored remotely and may be accessible by the device 400. In someembodiments, the correlations 414 may be static and predefined by, forexample, a manufacturer of the device. Alternately, the correlations 414may be configured by a user during set-up or use of the device.Alternately, the correlations 414 may be automatically modified by thedevice itself. As an example, the processor 404 may be configured to“learn” new correlations through pattern recognition or other means.Other types of correlations are possible as well.

The device 400 may include elements instead of and/or in addition tothose shown.

FIGS. 5 a-5 e show an example application of an example systemimplementing motion-detection logic. In particular, FIGS. 5 a-5 eillustrate a process of a processor executing motion-detection logic todetect a predetermined motion of the detector and using function logicto trigger a function upon detection of the predetermined motion. Inthis example, the motion-detection logic monitors and analyzes theimages seen by the detector to determine that the detector has undergonea predetermined motion in the pattern of the letter “J”. The imagesshown in the figures are merely illustrative and are not intended to belimiting in any way. In some contexts, the process described below maybe referred to as “vector displacement.”

In FIG. 5 a, the detector is located at a first position 502. While atposition 502, the detector records one or more images including thefirst image 504. As shown, the first image 504 includes three objects: awindow 510, a table 512, and a chair 514. In order to detect motion ofthe detector, the processor may execute motion-detection logic to selectone or more reference points within the first image 504 to be used inthe motion-detection process. For example, the processor may select as areference point 516 the bottom-right-hand corner of the window 514. Theprocessor may determine a position of the reference point 516 within thefirst image 504, such as by determining a set of coordinates to describethe position. As shown, the reference point 516 has a verticalcoordinate 506 and a horizontal coordinate 508.

In FIG. 5 b, the detector has moved to a second position 518. While atposition 518, the detector records one or more images including thesecond image 520. As shown, the second image 520 includes the same threeobjects 510, 512, 514 as the first image 504, but as a result of themovement of the detector from the first position 502 to the secondposition 518, the objects 510, 512, 514 have each shifted to a newposition within the second image 520. In particular, the reference point516 (the bottom-right-hand corner of the window) has shifted to a newposition within the second image 520. The processor may determine thenew position of the reference point 516 by determining a second set ofcoordinates to describe the position. As shown, the reference point 516now has a vertical coordinate 522 and a horizontal coordinate 524.

The processor may compare the first set of coordinates (506 and 508)with the second set of coordinates (522 and 524) to detect motion of thedetector. For example, by comparing the vertical coordinates 506 and522, the processor may determine that the reference point 516 has movedupwards in the second image 520 as compared to the first image 504. Fromthis, the processor may detect that the detector has moved downward inspace. Additionally, by comparing the horizontal coordinates 508 and524, the processor may determine that the reference point 516 has notmoved (or has moved very little) in a horizontal direction in the secondimage 520 as compared to the first image 504. From this, the processormay detect that the detector has not moved (or has moved very little) tothe left or right in space.

In FIG. 5 c, the detector has moved to a third position 526 where thedetector records one or more images including the third image 528. Asdescribed above, the processor may determine a third set of coordinatesto describe a new position of the reference point 516 within the thirdimage 528. As shown, the reference point 516 now has a verticalcoordinate 530 and a horizontal coordinate 532.

The processor may compare the third set of coordinates to one or both ofthe first set of coordinates and the second set of coordinates. Based onthe comparison(s), the processor may determine that the reference point516 has moved upwards and to the right in the third image 528 ascompared to the second image 520. From this, the processor may detectthat the detector has moved downwards and to the left in space.

In FIG. 5 d, the detector has moved to a fourth position 534 where thedetector records one or more images including the fourth image 536. Asdescribed above, the processor may determine a fourth set of coordinatesto describe a new position of the reference point 516 within the fourthimage 536. As shown, the reference point 516 now has a verticalcoordinate 538 and a horizontal coordinate 540.

The processor may compare the fourth set of coordinates to one or moreof the first set of coordinates, the second set of coordinates, and thethird set of coordinates. Based on the comparison(s), the processor maydetermine that the reference point 516 has moved downward and to theright in the fourth image 536 as compared to the third image 528. Fromthis, the processor may detect that the detector has moved upwards andto the left in space.

While four positions of the detector are shown, it is to be understoodthat the same principle of motion detection may be applied to any numberof positions. In the context of this example, there may be one or morepositions before the first position 502, after the fourth position 534,and/or between each of the first, second, third, and fourth positions ofthe detector. In detecting motion of the detector, the processor mayconsider one or more reference positions and/or sets of coordinatesdetermined from any additional positions.

In this manner, the processor may execute the motion-detection logic todetect motion of the detector. The processor may additionally executethe motion-detection logic to detect one or more predetermined motionsamong the detected motion of the detector. In the example, the processormay detect that the motion of the detector from the first position 502to the fourth position 534 is the predetermined motion in the shape of a“J”. Other predetermined motions are possible as well.

The processor may be additionally configured to look up thepredetermined motion (“J”) in a set of correlations in order todetermine a function. In the example, the predetermined motion may becorrelated with the function of entering the character “J” into anapplication running on a display.

In FIG. 5 e, an application 542 is shown running on a display 544. Upondetecting the predetermined motion (“J”), the processor may trigger thecorrelated function so that the character “J” 546 is entered in theapplication 542 running on the display 544.

The example shown in FIGS. 5 a-5 e is merely illustrative and is notmeant to be limiting. Many other devices, predetermined motions,functions, and correlations between predetermined motions and functionsare possible beyond those shown in FIGS. 5 a-5 e.

4. Third Example Device

While the foregoing has focused on devices that include only a singledetector, in some embodiments a device may be equipped with two or moredetectors. The inclusion of the additional detector(s) allows fordetection of not only the individual motions of the two or moredetectors, but also relative motion among the detectors.

FIG. 6 shows an embodiment of an example device with more than onedetector. As shown, the device 600 includes a first detector 602, asecond detector 604, a processor 606, and data storage 608 in which isstored stabilizing logic 610, relative-motion-detection logic 612,function logic 614, and correlations 616. The device 600 additionallyincludes an output interface 618. The elements of the device 600 areshown connected together by a system bus or other mechanism 620.

It is to be understood that, while each of the first detector 602, thesecond detector 604, the processor 606, the data storage 608, thestabilizing logic 610, the relative-motion-detection logic 612, thefunction logic 614, the correlations 616, and the output interface 618are shown to be integrated into the device 600, the device 600 may, insome embodiments, be made up of multiple devices among which theelements of device 600 are distributed, as described above.

The first detector 602 may be any detector configured to record a seriesof images of an environment. The first detector 602 may be, for example,any of the detectors described above.

Similarly, the second detector 604 may be any detector configured torecord a series of images of an environment. The second detector 604 maybe, for example, any of the detectors described above.

The processor 606 may be or may include one or more general purposeprocessors and/or dedicated processors. The processor 606 may beconfigured to execute one or more sets of logic stored in the datastorage 608.

In particular, the processor 606 may be configured to executestabilizing logic 610 to stabilize the series of images, as describedabove.

Additionally, the processor 606 may be configured to executerelative-motion-detection logic 610 to detect one or more motions ofeach of the first detector 602 and the second detector 604. Theprocessor 606 may be further configured to execute therelative-motion-detection logic 606 to detect, based on the detectedmotions of each detector, one or more predetermined relative motions ofthe first detector 602 and the second detector 604. The processor 606may be further configured to execute the relative-motion-detection logic612 to detect a speed of the detected motions and relative motions.

In some embodiments, the predetermined relative motions may includemovement in opposite directions, movement in the same direction, norelative movement, or combinations thereof. In some embodiments, themovements may include linear motions, such as upward motion, downwardmotion, leftward motion, rightward motion, forward motion, backwardmotion, and no motion. In some embodiments, the movements may includecircular motions, such as arcs and curves. In some embodiments, themovements may include a combination of motions, such as back-and-forthand start-stop-start motions. In some embodiments, the movements mayinclude one or more shapes and/or characters, such as letters of thealphabet. In some embodiments, the predetermined relative motions mayinclude known movements of a hand, such as American Sign Language ortyping on a standard QWERTY keyboard. Other predetermined motions arepossible as well. An example of detecting a predetermined relativemotion is described below in connection with FIGS. 7 a-7 b.

In some embodiments, the processor 606 may detect that neither of thefirst detector 602 and the second detector 604 are moving (or are movingvery little). In these embodiments, the detected relative motion may be“no relative movement.” For example, a user of the device 600 may beusing both the first detector 602 and the second detector 604 to pointat a particular object or in a particular direction.

In these embodiments, the processor 606 may be further configured to useone of the detectors, such as the first detector 602, to detect anarrower view, and to use the other of the detectors, such as the seconddetector 604, to detect a wider view. The processor 606 may be furtherconfigured to compare the narrower and wider views to detect an objectthat is approximately centered in each of the narrow and wide views. Thedetected object may be the object at which the user of the device 600 ispointing.

Additionally, the processor 606 may be configured to determine adirection, such as a cardinal direction, in which the user of the device600 is pointing. In some embodiments, this may involve detecting anobject that is approximately centered in each of the narrow and wideviews and determining a cardinal direction in which the user would haveto move to reach the detected object. To this end, the device 600 mayadditionally include a compass or other directional sensor.

The processor 606 may be further configured to execute function logic614 to trigger one or more functions upon detection of particularpredetermined relative motions. To this end, the function logic 614 maybe executable by the processor to look up one or more detectedpredetermined relative motions in a set of correlations, such ascorrelations 616. Correlations 616 may be correlations between aplurality of predetermined relative motions and a plurality offunctions. By looking up the one or more detected predetermined relativemotions in the correlations 616, the processor 606 may determine one ormore functions that are correlated with the predetermined relativemotion(s). As an example, if the detected predetermined relative motionis “no relative motion,” as described above, the one or more correlatedfunctions may include performing an image or text search on the detectedobject, or informing the user of the device 600 of the cardinaldirection. Other examples are possible as well.

Upon determining one or more functions that are correlated with thepredetermined motion(s), the processor 606 may trigger one or more ofthe one or more functions by sending instructions to the display via theoutput interface 618. To this end, the output interface 618 may becommunicatively coupled to the display via one or more of a wired and awireless link. An example of triggering a function based on detection ofa predetermined relative motion is described below in connection withFIGS. 7 c-7 d.

While the correlations 616 are shown to be stored in the data storage608 of the device 600, in some embodiments the correlations 616 may bestored remotely and may be accessible by the device 600. In someembodiments, the correlations 616 may be static and predefined by, forexample, a manufacturer of the device. Alternately, the correlations 616may be configured by a user during set-up or use of the device.Alternately, the correlations 616 may be automatically modified by thedevice itself. As an example, the processor 606 may be configured to“learn” new correlations through pattern recognition or other means.Other types of correlations are possible as well.

In some embodiments, the processor 606 may be configured to execute oneor more additional sets of logic stored in the data storage 608.

The device 600 may include elements instead of and/or in addition tothose shown.

FIGS. 7 a-7 d show an example application of an example system detectingrelative motion with two detectors. In particular, FIGS. 7 a-7 dillustrate a process of a processor executing relative-motion-detectionlogic to detect a predetermined relative motion of a first detector anda second detector and using function logic to trigger a function upondetection of the predetermined relative motion. The images shown in thefigures are merely illustrative and are not intended to be limiting inany way.

In FIG. 7 a, a first detector is shown to be located at a first position702. The first detector may record one or more images while located atthe first position 702, as described above. Similarly, a second detectoris shown at a first position 704. The second detector may record one ormore images while located at the first position 704, as described above.

Based on the image(s) recorded by each of the detectors, the processormay determine that the first detector is located to the left (as shown)of the right detector.

In FIG. 7 b, the first detector is shown to have moved to a secondposition 706. The first detector may record one or more images whilelocated at the second position 706, as described above. Additionally, aprocessor may compare one or more images recorded at the first position702 with one or more images recorded at the second position 706 todetect motion of the first detector, as described above. In the exampleshown, the processor may detect that the first detector has moved to theleft in space.

Also in FIG. 7 b, the second detector is shown to have moved to a secondposition 708. The second detector may similarly record one or moreimages while located at the second position 708, and the processor maycompare one or more images recorded at the first position 704 with oneor more images recorded at the second position 708 to detect motion ofthe second detector, as described above. In the example shown, theprocessor may detect that the second detector has moved to the right inspace.

The processor may thus detect motion of each of the first detector andthe second detector. By comparing the detected motion of the firstdetector with the detected motion of the second detector, the processormay determine a relative motion of the first detector and the seconddetector. In the example shown, the processor may determine that thefirst detector has moved to the left and the second detector has movedto the right. Because the processor previously determined that the firstdetector was located to the left of the second detector, the processormay conclude that the first detector and the second detector havehorizontally moved apart from one another, as shown. “Horizontallymoving apart” may be a predetermined relative motion.

In some embodiments, the processor may be further configured to comparethe detected motion of the first detector and the detected motion of thesecond detector with a statistical model in order to detect thepredetermined relative motion. As an example, the processor maydetermine that the first detector and the second detector movedhorizontally apart as well as vertically apart. By comparing thedetected motions of the first and second detectors with a statisticalmodel, the processor may determine that the most likely predeterminedrelative motion is “horizontally moving apart”. In some embodiments, oneor more statistical models may be stored in data storage at the device.Alternately or additionally, one or more statistical models may bestored remotely and may be accessible by the device. Such statisticalmodels may be static and predefined by, for example, a manufacturer ofthe device. Alternately or additionally, the statistical models may becontinuously updated based on user behavior or other feedback from theprocessor. Other statistical models are possible as well.

In some embodiments, the processor may additionally be configured todetermine a first distance between the first position 702 of the firstdetector and the first position 704 of the second detector based on oneor more of the images recorded by each of the detectors at the firstpositions and to determine a second distance between the second position706 of the first detector and the second position 708 of the seconddetector based on one or more of the images recorded by each of thedetectors. By comparing the first distance and the second distance, theprocessor may determine whether the first detector and the have movedtowards one another or away from one another. In some embodiments, thefirst distance and the second distance may be measured in known units ofsome kind, such as millimeters, and a quantitative comparison of thefirst and second positions may be made. Alternately, the distances maybe measured in arbitrary units and a qualitative comparison (e.g.,closer or further) of the first positions and the second positions maybe made.

In some embodiments, the processor may be further configured todetermine a speed of the detected relative motion of the first detectorand the second detector.

While only two positions of the first detector and two positions of thesecond detector are shown, it is to be understood that the sameprinciple of relative-motion detection may be applied to any number ofpositions of the first detector and/or second detector.

Thus, the processor may execute the relative-motion-detection logic todetect relative motion of the first detector and the second detector.The processor may additionally execute the relative-motion-detectionlogic to detect one or more predetermined relative motions among thedetected relative motions of the first detector and the second detector.In the example, the processor may detect that the relative motion of thefirst detector and the second detector (horizontally moving apart fromone another) is a predetermined relative motion.

The processor may be additionally configured to look up thepredetermined relative motion (“horizontally moving apart”) in a set ofcorrelations in order to determine a function. In the example, thepredetermined relative motion may be correlated with the function ofzooming in on a display.

In embodiments where one or more distances are determined betweenpositions of the first detector and the second detector, thepredetermined relative motion may be correlated with more than onefunction, and the processor may select between the functions based onthe one or more distances. For example, the predetermined motion may becorrelated with both a small zoom-in function (e.g., 125%) and a largezoom-in function (e.g., 200%). In order to select between the smallzoom-in function and the large zoom-in function, the processor maydetermine a difference between the one or more distances (e.g., adifference between the first distance and the second distance) and maycompare the difference to a predetermined threshold. In the example, ifthe difference is less than or equal to the predetermined threshold, thesmall zoom-in function may be selected, while if the difference isgreater than the predetermined threshold the large zoom-in function maybe selected. Other examples are possible as well.

In embodiments where a speed of the relative motion of the firstdetector and the second detector is determined, the predeterminedrelative motion may be correlated with more than one function, and theprocessor may select between the functions based on the determinedspeed. For example, the processor may select between a small zoom-infunction and a large zoom-in function based on a comparison of thedetermined speed to a predetermined threshold. Other examples arepossible as well.

FIG. 7 c shows the display 710 before the processor detects thepredetermined relative motion. Based on detection of the predeterminedmotion, the processor may trigger the correlated function. In theexample, the processor may trigger a page zoom (zoom in) on the display,as shown in FIG. 7 d.

The example shown in FIGS. 7 a-7 d is merely illustrative and is notmeant to be limiting. Many other devices, predetermined relativemotions, functions, and correlations between predetermined relativemotions and functions are possible beyond those shown in FIGS. 7 a-7 d.As an example, one or more additional detectors may be used record atleast one additional series of images. In this example, the processormay additionally use the relative-motion-detection logic to detect atleast one predetermined relative motion of the at least one additionaldetector, the first detector, and the second detector by comparing atleast one image in the first series of images with at least one image inthe second series of images and at least one image in the at least oneadditional series of images. Other examples are possible as well.

5. First Example Method

FIG. 8 is an example block diagram of a method of operating a device, inaccordance with at least some embodiments described herein. Method 800shown in FIG. 8 presents an embodiment of a method that, for example,could be used with systems and devices described herein. Method 800 mayinclude one or more operations, functions, or actions as illustrated byone or more of blocks 802-810. Although the blocks are illustrated in asequential order, these blocks may also be performed in parallel, and/orin a different order than those described herein. Also, the variousblocks may be combined into fewer blocks, divided into additionalblocks, and/or removed based upon the desired implementation.

In addition, for the method 800 and other processes and methodsdisclosed herein, the flowchart shows functionality and operation of onepossible implementation of present embodiments. In this regard, eachblock may represent a module, a segment, or a portion of program code,which includes one or more instructions executable by a processor forimplementing specific logical functions or steps in the process. Theprogram code may be stored on any type of computer readable medium, forexample, such as a storage device including a disk or hard drive. Thecomputer readable medium may include a non-transitory computer readablemedium, for example, such as computer-readable media that stores datafor short periods of time like register memory, processor cache andRandom Access Memory (RAM). The computer readable medium may alsoinclude non-transitory media, such as secondary or persistent long termstorage, like read only memory (ROM), optical or magnetic disks,compact-disc read only memory (CD-ROM), for example. The computerreadable media may also be any other volatile or non-volatile storagesystems. The computer readable medium may be considered a computerreadable storage medium, a tangible storage device, or other article ofmanufacture, for example.

In addition, for the method 800 and other processes and methodsdisclosed herein, each block in FIG. 8 may represent circuitry that iswired to perform the specific logical functions in the process.

As shown, the method 800 begins at block 802 where a wearable detectoris used to record a series of images of an environment. The environmentmay be, for example, the environment surrounding a user. In someembodiments, the environment may be an electronic display. Otherenvironments are possible as well.

At block 804, the device stabilizes the series of images recorded by thedetector to produce a stabilized series of images, as described above.

At block 806, the device determines at least one predetermined motion bycomparing at least two images in the stabilized series of images. Asdescribed above, the device may execute motion-detection logic to detectthe at least one predetermined motion.

At block 808, the device maintains correlations between a plurality ofpredetermined motions and a plurality of functions. In some embodiments,the correlations may be static and predefined by, for example, amanufacturer of the device. Alternately or additionally, thecorrelations may be configured by a user during set-up or use of thedevice. Alternately or additionally, the correlations may beautomatically modified by the device itself, such as by “learning” newcorrelations through pattern recognition or other means. Othercorrelations are possible as well.

At block 810, the device identifies the at least one predeterminedmotion in the correlations so as to determine at least one functionassociated with the at least one predetermined motion. In someembodiments, the device may then trigger the at least one function.

In some embodiments, the at least one predetermined motion may becorrelated with more than one function. In these embodiments, the devicemay trigger each of the functions, may trigger some of the functions, ormay trigger only one of the functions. The device may select among thefunctions randomly, or the device may consider other information indetermining which and how many functions to trigger. As an example, thedevice may consider previous functions triggered by the device. Asanother example, the device may perform image recognition on the seriesof images in order to recognize one or more objects in the series ofimages and may consider one or more of the recognized objects. As stillanother example, the device may detect a speed or distance of themotion, as described above, and may consider one or both of the speed ordistance. Other examples are possible as well.

The method 800 may be carried out periodically, continuously, as needed,as triggered, or in another manner.

6. Second Example Method

FIG. 9 is an example block diagram of another method of operating adevice, in accordance with at least some embodiments described herein.Method 900 shown in FIG. 9 presents an embodiment of a method that, forexample, could be used with systems and devices described herein. Method900 may include one or more operations, functions, or actions asillustrated by one or more of blocks 902-912. Although the blocks areillustrated in a sequential order, these blocks may also be performed inparallel, and/or in a different order than those described herein. Also,the various blocks may be combined into fewer blocks, divided intoadditional blocks, and/or removed based upon the desired implementation.

As shown, the method 900 begins at block 902 where a first detector isused to record a first series of images of an environment. The firstseries of images may include one or more images, and the images may berecorded at a constant or varying rate. The rate may be preselected by,for example, a manufacturer of the device. Alternately or additionally,the rate may be selected or modified by a user of the device.Alternately or additionally, the device may automatically adjust therate based on one or more factors such as, for example, a determinedspeed of the first detector. Other examples are possible as well.

At block 904, a second detector is used to record a second series ofimages of the environment. The second series of images may include oneor more images, and the images may be recorded at a constant or varyingrate. The rate may be preselected by, for example, a manufacturer of thedevice, may be selected or modified by a user of the device, and/or maybe automatically adjusted by the device based on one or more factors.

In some embodiments, the first detector and the second detector mayrecord images at the same rate. In other embodiments, the first detectorand the second detector may each record images at a rate independent ofthe rate of the other. Other examples are possible as well.

At block 906, the device may detect at least one predetermined relativemotion of the first detector and the second detector. This may involvethe device comparing at least one image in the first series of imageswith at least one image in the second series of images, as describedabove. In some embodiments, prior to block 906, the device may stabilizeone or both of the first series of images and the second series ofimages to produce one or both of a first stabilized series of images anda second stabilized series of images, as described above. In theseembodiments, block 906 may involve the device comparing at least oneimage in the first stabilized series of images with at least one imagein the second stabilized series of images.

At block 908, the device may maintain correlations between a pluralityof predetermined relative motions and a plurality of functions. In someembodiments, the correlations may be static and predefined by, forexample, a manufacturer of the device. Alternately, the correlations maybe configured by a user during set-up or use of the device. Alternately,the correlations may be automatically modified by the device itself,such as by “learning” new correlations through pattern recognition orother means. Other correlations are possible as well.

At block 910, the device may look up the detected at least onepredetermined relative motion in the correlations so as to determine thefunction and, at block 912, the device may trigger the function. Asdescribed above, in some embodiments, the at least one predeterminedrelative motion may be correlated with more than one function. In theseembodiments, the device may trigger each of the functions, may triggersome of the functions, or may trigger only one of the functions.Selection among the functions may be random or may be based onadditional information, as described above.

The method 900 may be carried out periodically, continuously, as needed,as triggered, or in another manner.

In some embodiments, one or more of the above-described devices may befurther configured to store one or more series of recorded images.Alternately or additionally, the device(s) may be configured to transmitone or more series of recorded images to a remote device for storage. Insome embodiments, the stored series of images may be used by the device,the remote device, or another device to analyze long-term movement of adetector. An example application for such analysis may be diagnosingarthritis in a user's hand. Other applications are possible as well.

While various aspects and embodiments have been disclosed herein, otheraspects and embodiments will be apparent to those skilled in the art.The various aspects and embodiments disclosed herein are for purposes ofillustration and are not intended to be limiting, with the true scopeand spirit being indicated by the following claims.

1. A glove, comprising: a detector positioned on a fingertip of theglove, wherein the detector is configured to record a series of imagesof an environment; a processor configured to (i) stabilize the series ofimages to produce a stabilized series of images, (ii) determine at leastone predetermined motion of the detector by comparing at least twoimages in the stabilized series of images, and, (iii) based on the atleast one predetermined motion, select at least one function from aplurality of functions; and an output interface configured to triggerthe at least one function.
 2. The glove of claim 1, wherein the outputinterface being configured to trigger the at least one functioncomprises the output interface being configured to trigger the at leastone function to be carried out on a display.
 3. The glove of claim 1,wherein the display comprises at least one of a display that is wearableby a user and a display that is remote from the glove.
 4. A wearabledevice, comprising: a detector positioned on a fingertip of the wearabledevice configured to record a first series of images of an environment;a processor configured to (i) stabilize the first series of images toproduce a first stabilized series of images, (ii) determine at least onepredetermined motion of the detector by comparing at least two images inthe first stabilized series of images, and, (iii) based on the at leastone predetermined motion, select a first at least one function from aplurality of functions; and an output interface configured to transmitan output based at least in part on the first at least one function. 5.The wearable device of claim 4, further comprising an orientation sensorconfigured to sense motion of the detector.
 6. The wearable device ofclaim 5, wherein the processor is further configured to stabilize theseries of images based on the motion of the detector.
 7. The wearabledevice of claim 4, wherein the output interface is further configured totransmit the output to a display that comprises at least one of adisplay that is wearable by a user and a display that is remote from thewearable device.
 8. The wearable device of claim 4, wherein theprocessor being configured to, based on the at least one predeterminedmotion, select the first at least one function from a plurality offunctions comprises the processor being configured to look up the atleast one predetermined motion in a set of correlations so as todetermine the first at least one function.
 9. The wearable device ofclaim 4, wherein the output interface being configured to transmit anoutput based at least in part on the first at least one functioncomprises the output interface being configured to trigger the first atleast one function to be carried out on a display.
 10. The wearabledevice of claim 4, further comprising a second detector configured torecord a second series of images of the environment.
 11. The wearabledevice of claim 10, wherein the processor is further configured todetermine at least one predetermined relative motion of the firstdetector and the second detector by comparing at least one image in thefirst series of images with at least one image in the second series ofimages, and, based on the at least one predetermined relative motion,select a second at least one function from the plurality of functions.12. The wearable device of claim 11, wherein the output is further basedat least in part on the second at least one function.
 13. The wearabledevice of claim 10, further comprising at least one additional detectorconfigured to record at least one additional series of images.
 14. Thewearable device of claim 13, wherein the relative-motion-detectionmodule is further configured to detect at least one predeterminedrelative motion of the at least one additional detector, the firstdetector, and the second detector by comparing at least one image in thefirst series of images with at least one image in the second series ofimages and at least one image in the at least one additional series ofimages.
 15. A method, comprising: recording a first series of images ofan environment with a first wearable detector positioned on a fingertipof a wearable device; stabilizing the first series of images to producea stabilized first series of images; determining at least onepredetermined motion by comparing at least two images in the firststabilized series of images; maintaining a first set of correlationsbetween a plurality of predetermined motions and a first plurality offunctions; and identifying the at least one predetermined motion in thefirst set of correlations so as to determine a first at least onefunction associated with the at least one predetermined motion.
 16. Themethod of claim 15, further comprising triggering the first at least onefunction.
 17. The method of claim 15, further comprising: recording asecond series of images of the environment with a second wearabledetector; determining at least one predetermined relative motion of thefirst wearable detector and the second wearable detector by comparing atleast one image in the first series of images with at least one image inthe second series of images; maintaining a second set of correlationsbetween a plurality of predetermined relative motions and a secondplurality of functions; and identifying the at least one predeterminedrelative motion in the second set of correlations so as to determine asecond at least one function associated with the at least onepredetermined relative motion.
 18. The method of claim 17, furthercomprising triggering the second at least one function.
 19. The methodof claim 17, wherein at least one of the first at least one function andthe second at least one function comprises at least one of entering atleast one character in an application displayed by a display, zooming inon the display, zooming out on the display, scrolling on the display,panning on the display, moving an indicator on the display, selecting anobject displayed by the display, using image recognition to identify atleast one object in the environment, searching for information relatedto the at least on object, displaying information related to the atleast one object on the display, identifying a cardinal direction, anddisplaying the cardinal direction on the display.
 20. The method ofclaim 15, wherein the first at least one function comprises at least afirst function and a second function, and the method further comprises:selecting between the first function and the second function based onone or more of a determined speed of one of the first detector or thesecond detector, a determined distance moved by one of the firstdetector or the second detector, an object identified in at least oneimage in the series of images, and one or more previously-triggeredfunctions; and triggering the selected function.